What Makes a Problem Hard for XCS?

Despite two decades of work learning classi er systems researchers have had relatively little to say on the subject of what makes a problem diÆcult for a classi er system. Wilson's accuracy-based XCS, a promising and increasingly popular classi er system, is, we feel, the natural rst choice of classi er system with which to address this issue. To make the task more tractable we limit our considerations to a restricted, but very important, class of problems. Most signi cantly, we consider only single step reinforcement learning problems and the use of the standard binary/ternary classi er systems language. In addition to distinguishing several dimensions of problem complexity for XCS, we consider their interactions, identify bounding cases of diÆculty, and consider complexity metrics for XCS. Based on these results we suggest a simple template for ternary single step test suites to more comprehensively evaluate classi er systems.